Naval  Research  Laboratory 

Stennis  Space  Center,  MS  39529-5004 


NRL/MR/7320-15-9590 


Validation  Test  Report  for  the 

Arctic  Cap  Nowcast/Forecast  System 

as  a  Fractures/Leads  and  Polynyas  Product 


Julia  W.  Crout 

Vencore  Services  and  Solutions,  Inc. 
Reston,  Virginia 

Pamela  G.  Posey 

Ocean  Dynamics  and  Prediction  Branch 
Oceanography  Division 


May  26,  2015 


Approved  for  public  release;  distribution  is  unlimited. 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
0MB  No.  0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  this  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including 
suggestions  for  reducing  this  burden  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson  Davis  Highway, 
Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of 
information  if  it  does  not  display  a  currently  valid  0MB  control  number.  PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


1.  REPORT  DATE  (DD-MM-YYYY) 
26-05-2015 

2.  REPORT  TYPE 

Memorandum  Report 

3.  DATES  COVERED  (From  -  To) 

4.  TITLE  AND  SUBTITLE 

5a.  CONTRACT  NUMBER 

Validation  Test  Report  for  the  Aretie  Cap  Noweast/Foreeast  System  as  a 

Fraetures/Leads  and  Polynyas  Produet 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

0603207N 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

Julia  W.  Crout*  and  Pamela  G.  Posey 


5e.  TASK  NUMBER 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 


5f.  WORK  UNIT  NUMBER 

73-9066-04-5 

8.  PERFORMING  ORGANIZATION  REPORT 
NUMBER 


Naval  Researeh  Laboratory 

Oeeanography  Division 

Stennis  Spaee  Center,  MS  39529-5004 


NRL/MR/7320- 15-9590 


9.  SPONSORING  /  MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


10.  SPONSOR  /  MONITOR’S  ACRONYM(S) 


Spaee  &  Naval  Warfare  Systems  Command 
2451  Crystal  Drive 
Arlington,  VA  22245-5200 


SPAWAR 


11 .  SPONSOR  /  MONITOR’S  REPORT 
NUMBER(S) 


12.  DISTRIBUTION  /  AVAILABILITY  STATEMENT 


Approved  for  publie  release;  distribution  is  unlimited. 


13.  SUPPLEMENTARY  NOTES 

*Veneore  Serviees  and  Solutions,  Ine.,  11091  Sunset  Hills  Road,  Suite  200,  Reston,  VA  20190. 


14.  ABSTRACT 

This  doeument  serves  as  the  Validation  Test  Report  (VTR)  for  the  Naval  Researeh  Laboratory  (NRL)  Aretie  Cap  Noweast/Foreeast  System 
(ACNFS)  ability  to  eapture  and  prediet  sea  iee  areas  of  opening  Fraetures/Leads,  And  Polynyas  (FLAP).  ACNFS  ealeulates  motion  rates  (opening/ 
elosing,  ridging,  divergenee,  and  shear)  in  addition  to  numerous  other  produets  sueh  as  iee  eoneentration,  iee  thiekness,  and  iee  drift.  Aeombination 
of  these  parameters  is  used  to  produee  a  produet  of  opening  rates  that  represent  areas  of  FLAPs.  This  VTR  doeuments  a  series  of  eomparison  studies 
performed  using  ACNFS  strain  rate  fields  and  validates  the  opening  rate  produets  against  an  1 1  month  period  of  National  Iee  Center  (NIC)  FLAP 
messages  from  January  through  November  2012.  Foreeasted  ACNFS  fields  are  evaluated  for  the  period  of  February  through  June  2014  on  daily 
24  hours  through  7-day  foreeasts.  Additional  eomparisons  are  shown  for  ICEX-2014.  This  validation  assessment  was  performed  using  ACNFS 
with  some  limited  eomparisons  with  the  Global  Oeean  Foreeast  System  (GOFS  3.1).  GOFS  3.1  uses  the  same  oeean  model  eoupling  and  sea  iee 
model  as  ACNFS  and  is  expeeted  to  perform  similarly.  Overall,  ACNFS  along  with  GOFS  3.1  provided  a  reasonable  predietion  of  the  openings 
not  available  elsewhere  in  both  the  noweast  and  foreeast  mode  and  performed  better  than  persistenee. 


15.  SUBJECT  TERMS 

Sea  iee  foreeasting  system  HYCOM  Lead  opening  validation 

Lead  opening  CICE 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION 

OF  ABSTRACT 

18.  NUMBER 

OF  PAGES 

19a.  NAME  OF  RESPONSIBLE  PERSON 

Pamela  G.  Posey 

a.  REPORT 

Unelassified 

Unlimited 

b.  ABSTRACT 

Unelassified 

Unlimited 

c.  THIS  PAGE 

Unelassified 

Unlimited 

Unelassified 

Unlimited 

43 

19b.  TELEPHONE  NUMBER  (include  area 
code) 

(228)  688-5596 

1 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std.  Z39.18 


11 


Table  of  Contents 

1  Introduction . 1 

2  Descriptions . 2 

2.1  ACNFS  Opening  Rate . 2 

2.2  ACNFS  Accumulated  Openings . 5 

2.3  ACNFS  Opening  Rate  Contours . 7 

3  Validation . 8 

3.1  Thresholds . 9 

3 .2  Model  Best  Quality . 10 

3.2.1  FLAP  Message  Validation . 10 

3.2.2  RADARSAT  Model  Analysis . 1 1 

3.3  Forecasting . 12 

3.3.1  Reference  Hindcasts . 13 

3.3.2  Persistence  versus  Model  Forecast . 22 

3.3.3  U.S.  Navy  Arctic  Submarine  Laboratory  Ice  Exercise  (ICEX) . 23 

3.4  Polynyas . 30 

3.5  Seasonal  Variation . 32 

4  Summary  and  Recommendations . 35 

5  Acknowledgements . 36 

6  References . 37 

7  Acronyms . 38 


iii 


List  of  Figures 


Figure  1 :  Top:  Notional  FLAP  message  for  Arctic  region  north  of  Greenland  Sea.  Left:  MODIS  imagery  with  fracture  poly¬ 


lines  in  green,  polynya  in  magenta,  and  ice  edge  in  red.  Right:  ACNFS  opening  rate  valid  for  same  time  period . 1 

Figure  2:  Daily  maximum  (top)  and  mean  (bottom)  opening  rate  values . 3 

Figure  3:  Distribution  of  opening  rate  values.  Cells  with  0  opening  rate  are  not  included . 4 

Figure  4:  ACNFS  opening  rate  for  OOZ  January  05,  2012.  a)  default  scale,  0  -  max  scale,  b)  scale  0-15 %/day.  c)  scale  0- 

4%/day. . 4 

Figure  5:  April  20,  2012  ACNFS  opening  rate  (left)  and  MODIS  imagery  (right)  just  off  the  northern  coast  of  Greenland.  ..5 
Figure  6:  Equation  3  represented  pictorially  with  basin  wide  openings  for  May  19,  2014  accumulating  3  previous  days  of 

weighted  opening  rates  reduced  by  convergence  rates . 6 

Figure  7:  Regional  openings  for  March  19,  2014  accumulating  2  previous  days  of  weighted  opening  rates  reduced  by 
convergence  rates.  Top  right  panel  shows  NPP  VIIRS  contrast  satellite  image  valid  March  18,  2014  (courtesy  of  NRL  - 

Monterey) . 7 

Figure  8:  Clockwise  from  left:  VIIRS  imagery  of  Chukchi  Sea  January  13,  2014  (Courtesy  of  NRL/Monterey);  ACNFS 

opening  rate  for  same  date;  ACNFS  opening  rate  for  same  date  with  opening  rate  contour  overlaid . 8 

Figure  9:  ACNFS  opening  rate  plotted  at  different  scales.  Left:  0  to  the  maximum  opening  rate  of  1510  %/day.  Middle:  0  to 

50  %/day.  Top:  0  to  4  %/day. . 9 

Figure  10:  ACNFS  openings  greater  than  different  threshold  values.  Left:  opening  rate  >0.1  %/day.  Middle:  opening  rate  > 

0.5  %/day.  Top:  opening  rate  >1.0  %/day. . 10 

Figure  1 1 :  February  20  to  February  26,  2008  deformation  and  velocity  fields  from  RADARS  AT  processing  system  (left  6 

panels  -  RADARS  AT  and  the  ACNFS  model  fields  (right  6  panels) . 12 

Figure  12:  Forecast  and  Reference  opening  rate  plots  from  February  13,  2014  nowcast . 14 

Figure  13:  Forecast  and  Reference  ice  drift  plots  from  February  13,  2014  nowcast . 15 

Figure  14:  Forecast  and  Reference  opening  rate  plots  from  March  11,  2014  nowcast . 16 

Figure  15:  Forecast  and  Reference  ice  drift  plots  from  March  11,  2014  nowcast . 17 

Figure  16:  Forecast  and  Reference  opening  rate  plots  from  March  1 1,  2014  nowcast  for  Kara  Sea . 18 

Figure  17:  Forecast  and  Reference  air  stress  plots  from  March  1 1,  2014  nowcast  for  Kara  Sea . 19 

Figure  18:  Root  Mean  Square  Error  of  the  opening  rate  forecasts.  Forecast  time  periods  are  from  February  through  June 

2014 . 20 

Figure  19:  Contingency  plot  threshold  selection . 21 

Figure  20:  Daily  and  mean  accuracy,  sensitivity,  and  specificity  of  ACNFS  model  forecasts  out  to  7  days.  (February  through 

June  2014) . 22 

Figure  21:  Persistence  and  model  forecasts  RMS  Error  in  opening  rate.  (February  through  June  2014) . 23 

Figure  22:  Daily  and  mean  accuracy,  sensitivity,  and  specificity  of  ACNFS  model  forecasts  and  ACNFS  persistence  out  to  7 

days.  (February  through  June  2014) . 23 

Figure  23:  Beaufort  Sea  ACNFS  Air  stress  for  March  14,  15,  and  16,  2014.  ICEX  2014  ice  camp  general  region  outlined 

in  red . 24 

Figure  24:  Beaufort  Sea  MODIS  and  RADARSAT-2  mosaic  image  for  March  16,  2014 . 25 

Figure  25:  Beaufort  Sea  ACNFS  Opening  rate  nowcast  and  forecasts  for  March  16,  2014 . 26 

Figure  26:  Beaufort  Sea  ACNFS  Ice  velocity  6-day  out  forecast  for  March  20,  2014 . 27 

Figure  27:  Beaufort  Sea  ACNFS  Opening  rate  hindcast,  nowcast,  and  24-hour  forecast . 28 

Figure  28:  Beaufort  Sea  MODIS  mosaic  image  for  March  22,  2014.  Wind  and  ice  drift  vectors  overlaid . 29 

Figure  29:  Open  ocean  polynya  August  16,  2012.  Right:  MODIS  imagery  (NASA,  2014)  with  ACNFS  40%  concentration 
contour  overlaid  in  black.  Top,  left:  ACNFS  opening  rate  with  40%  concentration  contour.  Bottom,  left:  ACNFS  ice 

concentration . 30 

Figure  30:  VIIRS  Near-Constant  Contrast  (NCC)  imagery  (NRL-MRY,  2014)  over  Kara  Sea  from  April  15  through  April  22, 
2014 . 31 


Figure  3 1 :  ACNFS  Opening  rate  with  70%  ice  concentration  contour  in  black  for  Kara  Sea  from  April  15  through  April  22, 
2014 . 31 


Figure  32:  RMS  Error  of  the  opening  rate  forecasts.  Winter  months  are  shown  in  shades  of  blue.  Spring  months  are  shown 

in  shades  of  green . 32 

Figure  33:  Specificity,  Sensitivity,  and  Accuracy  classification  errors  of  opening  rate  forecasts.  Winter  months  are  shown  in 

shades  of  blue.  Spring  months  are  shown  in  shades  of  green . 33 

Figure  34:  Daily  number  of  ice  cells  with  concentration  greater  than  15% . 33 


IV 


List  of  Tables 


Table  1 :  FLAP  area  prediction  comparison  categories  totals  “V”  indicated  a  strong  match;  “?”  indicates  a  match  to  some 

degree,  and  “X”  indicates  a  poor  match . 11 

Table  2:  Contingency  table  used  for  forecasting  ice  openings . 21 

Table  3:  FLAP  area  prediction  comparison  categories  totals  broken  out  by  season . 34 

Table  4:  FLAP  area  skill  totals . 35 

Table  5:  Flap  area  forecast  skill  metrics  for  first  72  hours . 35 


V 


1  Introduction 


The  Naval  Research  Laboratory  (NRL)  Arctic  Cap  Nowcast/Forecast  System  (ACNFS)  provides 
nowcast  and  forecast  fields  of  ice  parameters  such  as  ice  concentration,  ice  thickness,  and  ice 
drift  which  have  been  validated  using  observational  data  sets  (Posey  et  ah,  2010).  ACNFS  also 
provides  ice  motion  strain  rate  fields  of  divergence,  shear,  and  opening  rate  which  can  be  used  to 
provide  an  indication  of  zones  of  ice  fracturing  of  leads  and  polynyas. 


The  National  Ice  Center  (NIC)  produces  a  Fractures/Leads  and  Polynyas  (FLAP)  product  which 
is  a  formatted  text  message  that  identifies  navigation  and  surfaceable  features  in  the  ice  over 
large  areas.  The  messages  contains  the  latitude/longitude  pairs  delineating  FLAPs,  as  well  as 
remarks  on  the  orientation,  and  ice  types  that  a  NIC  ice  analyst  generated  from  available 
imagery.  FLAP  messages  are  produced  for  non-routine,  special  operations  and  exercises 
(submarine  and  surface  ships)  disseminated  via  message  traffic  and  Submarine  Forces  broadcast. 
Figure  1  shows  a  notional  example  of  a  FLAP  message  built  from  Moderate  Resolution  Imaging 
Spectroradiometer  (MODIS)  imagery  and  the  corresponding  ACNFS  opening  rate  product. 


OVLY/  UNCLASSIFIED 

LINE/7/A/2/790000N/0900000E/784928N/0900000E/790000N/0780000E/790000N/0785040E784928N/0900000E/790000 
N/0780000E/790000N/0785040E 
ARMKS/100  BOUNDARY 
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ARMKS/POLYNYA 

RMKS/ICE  EDGE  ANALYSIS  FROM  3/09  MODIS.  ANALYSIS  FRC»1  3/09  RADARSAT  AND  MODIS 


Figure  1:  Top:  Notional  FLAP  message  for  Arctic  region  north  of  Kara  Sea.  Left:  MODIS  imagery  with  fracture 


poly -lines  in  green,  polynya  in  magenta,  and  ice  edge  in  red.  Right:  ACNFS  opening  rate  valid  for  same  time 


period. 


As  part  of  the  NIC  Operational  Evaluation  (OPEVAL)  of  ACNFS  (Helfrich,  2012),  the  NIC’s 
Operations  Department  (OPS,  N3)  performed  a  utilization  study  of  ACNFS  beyond  its  objective 
testing  of  ice  concentration,  thickness  and  drifts  to  compare  ACNFS  lead  opening  rate  to  the 
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daily  NIC  analysis.  The  ACNFS  opening  rate  was  rated  showing  “good”  potential  for  iee 
charting  and  as  “very  good”  potential  for  FLAP  and  annotated  imagery  special  support 
generation.  The  NIC  limited  comparisons  showed  that  the  ACNFS  provided  a  general 
understanding  for  regional  potential  formation  of  leads,  but  was  less  able  to  resolve  actual 
location  and  orientation.  Also  noted  in  the  NIC  OPS  evaluation  was  the  importance  in 
forecasting  lead  openings. 

Initial  comparisons  of  ACNFS  and  MODIS  imagery  studies  by  NRL  were  similar  to  NIC 
findings  in  their  OPEVAL  testing.  Whereas  the  scale  of  the  imagery  was  between  250  m  and  I 
km  and  often  a  mosaic  of  imagery;  the  scale  of  ACNFS  is  approximately  3.5  km.  While  the 
ACNFS  opening  rate  products  looked  realistic,  they  rarely  matched  the  fracture -to-fractures 
details  in  the  satellite  imagery.  Given  the  dynamic  nature,  temporal  and  spatial  scales,  and  the 
model  ice  rheology,  individual  fracture  of  opening  grid  cells  are  not  expected  to  be  resolved. 
The  model  did  well  depicting  relative  regions  of  fractures  which  are  similar  to  the  FLAP  areas 
identified  by  the  NIC  FLAP  messages.  As  shown  in  Section  3.2.1  the  model  does  well 
capturing  the  fracture  field  on  this  scale. 

This  Validation  Test  Report  documents  a  series  of  comparison  studies  performed  using  ACNFS 
strain  rate  fields  -  in  particular,  the  opening  rate  and  openings  based  on  an  accumulation  of 
opening  fields.  ACNFS  opening  rate  products  were  validated  against  an  11  month  period  of 
FLAP  messages  from  January  through  November  2012.  The  FLAP  messages  provided  reference 
data  to  validate  the  best  quality  two-day  hindcast  ACNFS  model  output.  Forecasted  ACNFS 
fields  are  also  evaluated  for  the  period  of  February  through  June  2014  on  daily  24  hour  through 
168  hour  forecasts. 


2  Descriptions 

2.1  ACNFS  Opening  Rate 

The  ACNFS  (Posey  et  al.  ,2010)  consists  of  the  Los  Alamos  Sea  Ice  Model,  the  Community  Ice 
Code  (CICE)  (Hunke  and  Lipscomb,  2004)  coupled  with  the  HYbrid  Coordinate  Ocean  Model 
(HYCOM)  (Metzger  et  al.,  2010),  with  daily  Special  Sensor  Microwave  Imager/Sounder 
(SSMES)  ice  concentrations  assimilated  through  Navy  Coupled  Ocean  Data  Assimilation 
(NCODA)  (Cummings  and  Smedstad,  2013).  Navy  Operational  Global  Atmospheric  Prediction 
System  (NOGAPS)  (Hogan  et  al.,  1991)  forcing  was  applied  daily  to  ACNFS  prior  to  13  March 
2013,  after  which  forcing  was  replaced  by  the  NAVy  Global  Environmental  Model  (NAVGEM) 
(Metzger  et  al.,  2013).  The  ACNFS  provides  daily  7-day  ice  products  forecasts  at  approximately 
a  3.5  km  resolution. 

CICE  uses  an  elastic  viscous  plastic  (EVP)  ice  rheology  (Hunke  and  Lipscomb,  2004)  to 
describe  the  ice  dynamics  and  compute  strain  rates.  It  incorporates  the  standard  ridging  scheme 
of  Thorndike  et  al.  (1975)  to  compute  the  rates  of  opening,  closing,  and  ridging  given  the  strain 
rates  (Lipscomb  et  al.,  2007).  ACNFS  calculates  the  divergence  (positive  values  are  divergence, 
negative  values  convergence)  and  shear  strain  rates  as  well  as  the  computed  opening  rate.  The 
strains  and  opening  rates  are  in  units  of  %/day.  Opening  rate  values  vary  greatly  between  0  (no 
opening)  to  more  than  1000%/day.  Mean  values  however  are  on  the  order  of  3  %/day,  with  most 
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Ma^^imum  opening  rate  (X/day) 


values  below.  Figure  2  shows  the  maximum  and  mean  of  the  daily  ACNFS  opening  rates  for  the 
time  period  of  2010  through  2013.  Figure  3  shows  the  distribution  of  the  opening  rate  values  for 
the  same  period. 


Day  Df  year 


Figure  2:  Daily  maximum  (top)  and  mean  (bottom)  opening  rate  values. 
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Figure  3;  Distribution  of  opening  rate  values.  Cells  with  0  opening  rate  are  not  included.  The  distribution  of  cells 
at  bin  0  are  greater  than  0.0  (but  less  than  0.25). 


Figure  4  (a-c)  shows  the  opening  rate  for  a  winter  day  plotted  with  several  color  scales.  Figure 
4a  shows  the  default  color  scaling  of  0  to  ~1000%/day.  By  using  this  larger  color  scale,  the 
openings  are  not  easy  to  identify  showing  no  openings  with  only  a  slight  hint  of  color  in  the 
Denmark  Strait.  Figure  4b  shows  that  by  reducing  the  scale  closer  to  the  average  values  (0- 
15%/day),  yields  delineated  fracture-like  features.  Figure  4c  shows  a  scale  of  0  to  4%/day  that 
was  used  for  most  visualization  of  the  opening  rates. 


a)  b)  c) 


Figure  4:  ACNFS  opening  rate  for  OOZ  January  05,  2012.  a)  default  scale,  0  -  max  scale,  b)  scale  0-15%/day.  c) 
scale  0-4%/day. 


In  preliminary  comparisons,  the  ACNFS  opening  rate  field  looked  reasonable  with  features  often 
resembling  satellite  imagery.  Figure  5  shows  an  example  of  a  typical  ACNFS  opening  rate 
product  illustrating  realistic  fractures  as  compared  to  satellite  imagery. 
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Figure  5:  April  20,  2012  ACNFS  opening  rate  (left)  and  MODIS  imagery  (right)  just  off  the  northern  coast  of 
Greenland. 


2.2  ACNFS  Accumulated  Openings 

The  daily  ACNFS  model  opening  rate  is  an  instantaneous  value.  It  provides  an  indication  of  how 
fast  an  opening  event  occurs  with  a  unit  of  percent  per  day.  Unlike  imagery,  which  shows 
openings  that  are  present,  an  opening  rate  is  an  indication  of  new  or  expanding  fracturing.  The 
opening  rate  field  does  not  reflect  ice  openings  from  previous  days,  unless  they  are  expanding  or 
are  large  enough  to  be  seen  in  the  ice  concentration. 

For  exercise/operations  planning  and  forecasting,  knowledge  of  the  timing  and  location(s)  of 
significant  fracturing  expected  to  occur  is  important.  But  for  daily  surface  ship  and  submarine 
navigation  and  surfacing,  knowledge  of  where  openings  are  present,  or  expected  is  important.  To 
simulate  the  existing  openings  and  the  leads  that  are  opening,  an  accumulation  of  opening  over 
the  course  of  a  few  days  is  computed.  The  model  does  not  determine  a  closing  rate,  but  it  does 
produce  a  convergence,  which  is  closely  related.  A  more  comparable  model  product  can  be 
produced  by  using  the  previous  3  days  from  the  time  of  interest,  with  opening  rates  being 
weighted  and  then  applying  any  convergence  afterwards. 

The  openings  for  a  day  of  interest,  d  is  computed  for  each  grid  cell  separately  as 


(1) 
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where  is  the  accumulated  opening  from  the  previous  day  and  given  by  the  recursive  formula 
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where  i  is  the  day  past  from  the  day  of  interest,  is  the  opening  rate,  is  the  convergence 
strain  rate,  and  ft)  ■  is  the  damping  weight  applied  to  the  daily  opening  rate.  Expanding  Equation 
1  for  a  three  day  accumulation  yields 
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Figure  6  illustrates  Equation  3  for  a  three  day  aeeumulation  of  openings  on  a  basin-wide  seale. 
Figure  7  illustrates  a  two  day  aeeumulation  of  openings  on  a  regional  seale  aeeompanied  with 
VIIRS  imagery  for  the  valid  day.  Weights  of  CO-  =  {0.8,  0.6,  0.4.  0.2,  0}  were  ehosen  for  i  =  dto 
d-4  to  reduee  the  influenee  of  previous  openings. 
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Figure  6:  Equation  3  represented  pictorially  with  basin  wide  openings  for  May  19,  2014  accumulating  3  previous 
days  of  weighted  opening  rates  reduced  by  convergence  rates. 
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Figure  7:  Regional  openings  for  March  19,  2014  accumulating  2  previous  days  of  weighted  opening  rates  reduced 
by  convergence  rates.  Top  right  panel  shows  NPP  VllRS  contrast  satellite  image  valid  March  18,  2014  (courtesy  of 
NRL  -  Monterey ). 


2.3  ACNFS  Opening  Rate  Contours 

To  assist  in  the  visualization  of  the  fracture  field  and  to  provide  methods  of  quantifying  the 
results,  the  opening  rate  can  be  smoothed  and  contoured.  With  the  ACNFS  providing  an 
indication  of  the  general  area  of  openings,  rather  than  individual  openings,  bounding  areas  of 
fractures  may  yield  more  reliable  results.  To  capture  the  area  of  fracturing  on  the  scales  of  the 
FLAP  message  areas,  the  contour  from  an  18  point  smoothing  fdter  was  used.  This  contour 
provided  the  most  consistent  fracture  grouping.  Figure  8  illustrates  this  contouring  with  an 
example  of  fractures  in  the  Chukchi  Sea.  The  two  model  plots  on  the  right  have  the  same 
opening  rates  plotted  with  contour  overlaid  on  the  lower  plot. 
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Figure  8:  Clockwise  from  left:  VIIRS  imagery  of  Chukchi  Sea  January  13,  2014  (Courtesy  ofNRL/Monterey); 
ACNFS  opening  rate  for  same  date;  ACNFS  opening  rate  for  same  date  with  opening  rate  contour  overlaid. 


3  Validation 

The  ACNFS  operational  sea  ice  model  is  evaluated  primarily  on  the  basis  of  its  ability  to  capture 
Naval  Ice  Center  (NIC)  identified  Fracture/Leads,  and  Polynyas  (FLAP).  The  NIC  FLAP  dataset 
consists  of  81  FLAP  messages  from  January  2012  through  November  2012.  The  FLAP  text 
messages  delineate  areas  of  openings  in  the  ice  on  a  tactical  scale,  hand-drawn  by  NIC  ice 
analysts  from  available  satellite  imagery  including  Envisat  and  RADARS AT-2  (SAR);  and 
MODIS  and  DMSP/OLS  (visible).  The  messages  can  cover  a  sub-region  of  the  Arctic  or  can  be 
basin  wide.  The  messages  contain  coordinates  of  poly-line  segments  identifying  fracture  area 
polygons  and  polynya  area  polygons.  If  predominant,  orientations  of  fractures  and/or  ice  types 
are  noted.  In  total,  227  fracture  polygons  and  37  polynya  polygons  are  identified.  Coordinates 
north  of  82°  were  given  in  transpolar  coordinates  (where  the  north  pole  (90/0)  is  centered  instead 
at  Greenwich  mean  time  and  the  equator)  and  converted  to  standard  Greenwich  coordinates. 

As  described  in  Section  2.1,  ACNFS  consists  of  3  components:  1)  ice  -  CICE,  2)  ocean  - 
HYCOM  and  3)  data  assimilation  -  NCODA.  The  model  provides  a  daily  7-day  forecast  at 
approximately  3.5  km  resolution.  For  the  FLAP  comparisons,  the  model  output  from  the  2-day 
hindcast  (best  quality)  is  used.  ACNFS  opening  rates  are  used  as  the  primary  source  of 
comparison  and  are  indicative  of  the  amount  of  fracturing  and  lead  distributions. 

The  CICE  model  has  also  been  implemented  into  the  Global  Ocean  Forecast  System  (GOFS  3.1) 
(Metzger  et  al,  2014).  GOFS  3.1  is  comprised  of  the  1/12°  global  HYCOM  and  the  NCODA 
system  of  which  ACNFS  domain  is  a  subset  and  is  likewise  forced  with  NAVGEM  atmospheric 
forcing  out  to  7  days.  GOFS  3.1  provides  nowcasts  and  forecasts  of  the  global  ocean 
environment  which  includes  three-dimensional  ocean  temperature,  salinity  and  current  structure, 
surface  mixed  layer,  the  location  of  mesoscale  features,  ice  concentration,  thickness  and  ice  drift. 
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GOFS  3.1  has  the  capability  to  provide  ice  forecasts  in  the  Southern  Hemisphere  as  well  as  the 
Northern  Hemisphere.  GOFS  3.1  opening  rate  products  are  also  validated  as  part  of  this 
validation  test  report.  GOFS  3.1  passed  validation  in  September  2014,  with  transition  to 
NAVOCEANO  scheduled  by  2014  year’s  end. 


3.1  Thresholds 


Several  thresholds  were  determined  to  best  capture  the  fracture  fields:  1)  the  scale  in  which  to 
view  the  plots,  2)  the  minimum  opening  rate  to  consider  an  opening,  and  3)  the  concentration  in 
which  to  consider  openings.  As  shown  in  Figure  2,  the  maximum  opening  rate  over  the  Arctic 
basin  on  any  given  day  is  between  1000  and  3000  %/day,  with  extremely  high  opening  rate 
values  isolated  generally  near  the  ice  edges.  Figure  9,  similar  to  Figure  4,  shows  the  opening  rate 
output  for  another  typical  Arctic  day  plotted  at  different  scales.  Plotted  to  the  maximum  for  this 
day  of  1510  %/day  (Figure  9,  left),  no  fracturing  is  evident.  When  scaled  to  a  much  lower  value 
of  50  %/day  (Figure  9,  middle),  only  a  few  openings  are  seen  along  the  ice  edge.  Ultimately  at  a 
scale  from  0  to  4  %/day  (Figure  9,  right),  the  openings  are  easily  viewable.  Reasonable  scales 
limits  were  shown  to  be  between  from  0  to  3  -  1 5  %/day.  The  NIC  and  NRL  agreed  to  use  a 
scale  from  0  to  4  %/day  for  the  opening  rate  validation. 


Figure  9:  ACNFS  opening  rate  plotted  at  different  scales.  Left:  0  to  the  maximum  opening  rate  of  1510  %/day. 
Middle:  0  to  50  %/day.  Top:  0  to  4  %/day. 


Opening  rate  values  have  a  minimum  of  0  %/day  indicating  no  opening,  but  almost  all  grid  cells 
have  an  opening  rate  component,  even  if  extremely  small.  Choosing  a  minimum  as  any  opening 
greater  than  0  (or  a  small  value)  is  not  representative  of  the  fracturing,  as  it  implies  fracturing 
throughout.  A  minimum  opening  rate  of  1  %/day  was  chosen  as  best  mirroring  fracture  regions 
at  the  scale  seen  in  imagery  and  in  a  typical  FLAP  messages.  Figure  10  illustrates  the 
consequences  of  the  minimum  opening  rate.  Cells  are  colored  green  for  opening  rates  greater 
than  the  threshold  and  cells  are  colored  red  for  opening  rates  less  than  the  threshold.  The  left 
plot  has  a  minimum  threshold  of  0.1  %/day.  Only  a  few  isolated  cells  have  values  less  than  the 
minimum.  The  middle  plot  has  a  minimum  threshold  of  0.5  %/day.  Some  opening  structure  is 
becoming  visible,  but  it  is  still  not  a  reasonable  indication  of  actual  ice  openings.  The  right 
graphic  shows  a  minimum  threshold  of  1  %/day. 
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Figure  10:  ACNFS  openings  greater  than  different  threshold  values.  Left:  opening  rate  >0.1  %/day.  Middle: 
opening  rate  >0.5  %/day.  Top:  opening  rate  >1.0  %/day. 


3.2  Model  Best  Quality 

The  operational  ACNFS  (run  at  1 8Z)  produces  a  daily  nowcast  and  7  day  forecast  at  time  OOZ 
from  the  previous  day's  24  hr  restart  file.  Every  day  the  system  reaches  back  72  hours  from  the 
nowcast  time  (18Z)  to  use  any  late  arriving  satellite  data.  As  a  result,  ACNFS  produces  three 
hindcasts:  time  OOZ  on  the  model  run  day,  a  1-day  hindcast,  and  a  2-day  hindcast.  The  2-day 
best  quality  hindcast  is  considered  the  model's  best  representation  of  the  actual  ice  conditions 
and  is  used  in  the  comparison  against  the  FLAP  messages 

3.2.1  FLAP  Message  Validation 

Ice  openings  were  examined  through  an  initial  qualitative  analysis  comparing  distributions  of 
areas  of  ACNFS  opening  rates  and  FLAP  messages.  For  each  message,  the  number  of  fractures 
and  polynyas  along  with  the  density  and  orientation  noted  and  the  satellite  imagery  used  in  the 
analysis  were  listed.  Comparison  metrics  were  compiled  for  each  message  fracture  area  noting 
the  model  agreement  category  as  strong  match,  partially  covered,  location  off,  subset  of  field,  or 
no  match.  Table  1  (shown  below)  presents  the  summary  statistics  comparison  categories  totals. 
The  first  three  columns  are  the  gross  number  of  fracture  regions  identified  in  the  FLAP  messages 
that  went  into  the  statistics.  The  rest  of  the  columns  are  percentages  of  the  fracture  regions. 
Matches  are  classified  as  strong  (“V”),  meaning  the  model  contained  fracturing  in  the  same 
location  as  the  FLAP  message  polygon;  to  some  degree  (“?”),  broken  down  further  into  off-set 
location,  partial  area  match,  subset  of  area  matched,  and  weak  openings  in  area;  and  poor  match 
(“x”).  The  “V/?”  column  is  the  sum  of  the  strong  and  to  some  degree  columns.  The  metrics 
suggest  that  the  ACNFS  accumulated  openings  were  an  improvement  over  the  instantaneous 
opening  rate  and  that  ACNFS  and  GOFS  3.1  performed  similarly.  In  all  cases,  the  models  did 
well,  capturing  between  88-97%  of  the  FLAP  fracture  regions,  with  ~  30%  as  a  strong  match. 
Using  the  accumulated  openings  improved  the  miss-rate  considerably. 
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Fracture  Regions 

Percent  of  Fracture  Regions 

✓  ?  X 

✓ 

? 

off-set  partial  subset  weak 

? 

✓  +  ? 

X 

ACNFS  opening  rate 

49  89  18 

31% 

5%  21%  22%  9% 

57% 

88% 

12% 

GOFS  opening  rate* 

42  85  33 

26% 

4%  21%  18%  10% 

53% 

79% 

21% 

ACNFS  accumulated  openings 

67  95  6 

40% 

3%  10%  40%  4% 

57% 

97% 

4% 

Table  1:  FLAP  area  prediction  comparison  categories  totals  “V”  indicated  a  strong  match;  “?”  indicates  a  match 
to  some  degree,  and  “x”  indicates  a  poor  match.  “?”  is  broken  down  further  into  ojf-set  location,  partial  area 
match,  subset  of  area  matched,  and  weak  openings  in  area,  f  *  GOFS  best  quality  is  a  1-day  hindcast.) 


There  are  intrinsic  limitations  in  the  assessment  of  the  model  performance  against  the  FLAP 
messages.  The  poly-lines  in  the  messages  are  subjective,  hand  drawn  borders  made  by  different 
analysts,  under  different  circumstances,  with  different  and  unknown  intelligence/tactical  needs, 
and  with  different  and  unknown  source  imagery.  If  a  model  opening  area  is  not  included  in  a 
FLAP  message,  there  is  no  way  to  know  whether  or  not  it  exists.  In  the  absence  of  consistently 
clear  imagery  or  other  ground  truth  data,  the  FLAP  messages  are  the  best  validation  data. 


3.2.2  RADARSAT  Model  Analysis 

Some  publicly  available  basin- wide  RADARSAT- 1  data  transformed  into  estimates  of  ice  motion 
and  deformation  by  NASA’s  RADARSAT  Geophysical  Processor  System  (RGPS)  (Lindsay  and 
Stem,  2003)  coincides  with  available  ACNFS  rans.  The  RGPS  and  ACNFS  products  are 
available  during  winter. 

The  RGPS  deformation  fields  were  generated  over  weekly  time  periods  rather  than  daily.  To 
compare  the  daily  model  output  with  the  weekly  data,  the  ACNFS  output  strain  rates  (opening 
rate,  divergence,  and  convergence)  were  accumulated  over  the  weekly  time  period  to  capture  all 
openings  (closings).  Ice  drift  u-  and  v-  components  along  with  a  calculated  magnitude  were 
averaged  over  the  time  period.  Typical  results  are  shown  in  Figure  11,  with  the  model  indicating 
openings  in  the  same  general  areas  as  the  RADARSAT  divergence,  but  without  the  same 
stmctures.  The  ACNFS  and  the  RADARSAT  ice  drift  motion  correlate  fairly  well  on  the  gross 
scale. 
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Figure  11:  February  20  to  February  26,  2008  deformation  and  velocity  fields  from  RADARSAT processing  system 
(left  6  panels  -  RADARSAT  and  the  ACNFS  model  fields  (right  6  panels).  Panels  indicate  the  following: 
RADARSAT:  top  left  -  divergence  ( + )/convergence  ( - ),  top  right  -  u-component  velocity,  middle  left  -  vorticity, 
middle  right  -  v -component  velocity,  bottom  left  -  shear,  bottom  right  -  ice  drift  magnitude.  ACNFS:  top  left  - 
maximum  convergence,  top  right  -  average  u-component  velocity,  middle  left  -  maximum  divergence,  middle  right  - 
average  v-component  velocity,  bottom  left  -  cumulative  opening  and  bottom  right  -  average  ice  drift  velocity. 


3.3  Forecasting 

Section  3.3  indicates  how  well  ACNFS  performed  in  capturing  the  known  ice  openings  in 
forecasting  the  opening  rate  out  for  7  days.  The  forecast  validation  covers  the  period  of  February 
2014  -  July  2014.  The  forecasts  were  validated  on  a  full  Arctic  scale  against  reference  2 -day 
hindcasts  and  compared  with  persistence.  On  a  regional  scale,  the  model  was  compared  against 
events  of  the  Commander,  Submarine  Forces  (COMSUBFOR)  ICEX  2014  exercise. 

The  validation  shows  that  the  ACNFS  forecasts  provide  value  added  over  persistence  even  out  to 

6  day,  performing  particularly  well  in  the  first  24  to  48  hours,  and  then  degrading  out  though  the 

7  days. 
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3.3.1  Reference  Hindcasts 


In  the  absence  of  ground  truth  data  (no  FLAP  messages)  during  the  time  period  that  model 
forecasts  were  available,  the  ACNFS  2-day  hindcasts  were  used  as  the  reference  data  for  the 
model  forecasts.  The  2-day  hindcast  is  considered  the  model's  best  representation  of  the  true  ice 
conditions. 


3.3.1.1  Visual  Inspection 

In  determining  the  forecast  skill  for  ACNFS  opening  rates,  several  weeks  of  forecasts  over 
several  months  were  examined,  both  on  the  full  Arctic  and  regional  scales. 

Figure  12  shows  a  sample  7-day  forecast  of  opening  rate  for  the  second  week  in  February  2014. 
The  reference  hindcasts  are  shown  directly  below  the  forecasts.  The  model  does  extremely  well 
through  the  first  24  hour  forecast  and  still  very  well  out  through  4  days.  By  the  7*  day,  the 
model  forecasted  opening  rate  product  has  degraded.  Figure  13  shows  the  corresponding  7-day 
forecast  of  ice  drift.  The  same  trend  is  clearly  seen  in  the  ice  drifts,  with  the  7*  day  drift  being 
drastically  different.  The  7*’’  forecast  day  was  often  inconsistent  due  to  the  model  7*  day  forcing 
not  being  available  at  run-time.  In  this  case,  the  model  reverts  to  climatological  forcing. 

Figure  14  is  the  forecast  for  a  month  later  in  March  2014.  In  this  case,  the  model  does  extremely 
well  out  through  4  days  and  very  well  out  though  the  6*.  As  with  the  previous  example,  by  the 
7*’’  day,  the  model  did  not  do  well.  The  corresponding  ice  drifts  are  shown  in  Figure  15.  The  ice 
drifts  were  well  captured  out  through  5  days. 

Examining  ACNFS  on  a  smaller  scale.  Figure  16  zooms  into  the  Kara  Sea  region  where  more 
fracture  details  can  be  seen  and  differences  are  more  visible.  The  model  did  very  well  out  to  3 
days,  but  for  the  4-7  day  forecasts,  the  model  forecast  did  not  do  as  well.  By  the  7*’’  day  the 
model  picked  the  fracturing  back  up.  A  look  at  the  forecasted  air  stress  (Figure  17)  reveals  why 
the  opening  rate  got  off  after  3  days,  the  air  forcing  was  very  different  for  4  through  7*’’  forecast 
days. 

The  samples  shown  are  typical  forecasts  and  representative  of  the  initial  correlation  and  then 
gradual  degradation  of  the  forecast. 
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Figure  12:  Forecast  and  Reference  opening  rate  plots  from  February  13,  2014  nowcast. 
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Figure  14:  Forecast  and  Reference  opening  rate  plots  from  March  11,  2014  nowcast. 
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Figure  15:  Forecast  and  Reference  ice  drift  plots  from  March  11,  2014  nowcast. 
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Figure  16:  Forecast  and  Reference  opening  rate  plots  from  March  11,  2014  nowcast  for  Kara  Sea. 
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Figure  1 7:  Forecast  and  Reference  air  stress  plots  from  March  11,  2014  nowcast  for  Kara  Sea. 


3.3.1.2  RMSE 

Validation  is  also  performed  by  analyzing  statistically  the  agreement  between  the  forecasts  and 
the  reference  hindcasts.  The  Root  Mean  Square  Error  (RMSE)  was  computed  for  the  sample 
forecasts  and  is  shown  in  Figure  18.  The  hindcasts  are  included  in  the  computations  and  are 
shown  as  negative  forecast  days.  The  reference  hindcast  is  shown  as  -3  days  with  an  error  =  0. 
The  daily  nowcast  analyses  are  shown  at  day  0  with  the  forecast  days  to  the  right.  The  forecast 
clearly  performs  better  within  a  shorter  forecast  period  and  degrades  with  time.  The  RMSE 
growth  rate  is  consistent  between  forecasts,  with  only  a  few  exceptions  and  is  highest  between 
the  nowcast  and  the  1-day  forecast.  After  the  1-day,  the  forecasts  have  a  decreasing  slope, 
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becoming  flat  by  the  7*  day. 
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Figure  18:  Root  Mean  Square  Error  of  the  opening  rate  forecasts.  Forecast  time  periods  are  from  February 
through  June  2014. 


3.3.1.3  Contingency  Validation 

Since  observed  fracturing  consists  of  the  presence  or  absence  of  a  fracture  in  an  area,  the 
accuracy  is  assessed  by  comparing  the  forecasts  with  a  binary  classification  of  opening  and  non¬ 
opening  areas  of  the  reference  analysis  for  the  forecast  time.  The  comparison  of  reference  and 
forecast  model  results  classified  into  2  classes  is  represented  through  contingency  tables.  Table  2 
shows  the  contingency  table  used  to  score  the  forecast  performance  against  the  reference 
hindcast. 
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2;  Contingency  table  used  for  forecasting  ice  openings. 


To  reduce  the  opening  rate  values  to  an  opening/non-openings  classification,  a  cut-off  value  is 
needed.  As  discussed  in  Section  3.2.1,  the  model  is  used  to  provide  an  area  of  openings  rather 
than  openings  for  individual  grid  cells.  To  closely  match  the  scale  of  the  fracture  areas  in  the 
FLAP  messages,  the  same  smoothing  that  was  applied  to  capture  the  fracture  fields  is  used  in  the 
contingency  metrics.  After  experimenting  with  filters  and  cut-off  values,  a  cut-off  opening  rate 
value  of  0.5  %/day  applied  after  the  18  point  smoothing  filter  was  selected  to  best  represent  the 
FLAP  contours.  Figure  19  illustrates  the  effect  of  the  cutoff  value.  The  0.25  %/day  threshold 
yields  a  basin  almost  entirely  covered  by  fracture  areas.  On  the  other  end,  a  1.0  %/day  threshold 
yields  a  basin  almost  entirely  void  of  fracture  areas.  A  0.5  %/day  threshold  yielded  the  most 
reasonable  fracture  areas. 


Figure  19:  Contingency  plot  threshold  selection. 


By  combining  correctly  and  incorrectly  classified  openings  and  non-openings,  quality  measures 
can  be  derived.  To  analyze  ACNFS  FLAP-like  product  capability,  contingency  plots  were  made 
and  statistical  quality  metrics  of  their  overall  accuracy,  sensitivity,  and  specificity  accumulated. 

Accuracy  =  (True  Positive  +True  Negative)/  All  Positive  and  Negative. 

Sensitivity  =  True  Positive  /  (True  Positive  +  False  Negative). 

Specificity  =  True  Negative/  (True  Negative  +  False  Positive). 
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Preliminary  results  (February  through  June  2014)  are  very  promising  and  show  that  the  model 
does  very  well  in  the  first  1  -  4  forecast  days  with  generally  over  80%  sensitivity  and  then  tapers 
off  slowly  from  there  as  opposed  to  persistence  which  only  has  about  50%  sensitivity  for  the  24 
hours  forecast.  Figure  20  plots  the  forecast  contingency  accuracy  metrics  out  to  7  forecast  days. 
Individual  forecasts  are  shown  in  grey  and  the  mean  of  the  forecasts  in  black. 

Sensitivity  is  a  measure  of  how  well  a  test  was  at  detecting  a  condition.  A  high  sensitivity  in  this 
context  implies  that  the  model  did  well  predicting  opening  cells.  The  sensitivity  is  very  high  for 
the  nowcast  and  drops  off  quickly,  fairly  linearly  with  each  day  further  out.  The  often 
inconsistent  7*’’  forecast  day  is  due  to  the  model  7*  day  forcing  not  being  available  at  run-time. 
In  this  case,  the  model  reverts  to  climatological  forcing. 

Specificity  is  a  measure  of  how  well  a  test  correctly  identifies  the  absence  of  a  condition.  A  high 
specificity  implies  that  the  model  did  well  predicting  non-opening  cells.  The  specificity  is  very 
high  for  the  nowcast  and  24  hour  forecast  and  drops  quickly  for  the  next  48  hours.  After  that  a 
slower  drop-off  in  specificity  is  seen. 


Figure  20:  Daily  and  mean  accuracy,  sensitivity,  and  specificity  ofACNFS  model  forecasts  out  to  7  days.  (February 
through  June  2014) 


3.3.2  Persistence  versus  Model  Forecast 

To  provide  context  to  the  performance  of  the  model  forecast  predictions,  the  ACNFS  opening 
rate  forecasts  were  compared  against  persistence.  Persistence  assumes  no  skill  in  forecasting. 
The  last  known  condition,  in  this  case  the  reference  hindcast  opening  rates  for  the  day  prior  to 
the  nowcast  day,  is  held  constant  for  all  forecast  days.  The  same  statistics  used  to  analyze  the 
model  forecasts  were  calculated  for  the  persistence  forecasts.  As  expected,  the  RMSE,  accuracy, 
sensitivity,  and  specificity  statistics  show  that  the  ACNFS  forecasts  performed  better  than 
persistence.  This  improvement  extended  out  to  6  forecast  days.  By  the  7  day  forecast, 
persistence  had  almost  caught  up  to  the  model  forecasts.  Figure  21  and  Figure  22  shows  the 
improvements  in  model  forecasting  relative  to  persistence.  Individual  forecasts  and  their 
corresponding  persistence  are  shown  in  matching  colors.  Mean  persistence  and  model  forecasts 
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are  shown  in  black. 
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Figure  21:  Persistence  and  model  forecasts  RMS  Error  in  opening  rate.  (February  through  June  2014) 


Forecast  and  Persistence  versus  Best  Analysis  -  Accuracy 
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Forecast  and  Persistence  versus  Best  Analysis  Sensitivity 
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Forecast  and  Persistence  versus  Best  Analysis  Specificity 
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Figure  22:  Daily  and  mean  accuracy,  sensitivity,  and  specificity  ofACNFS  model  forecasts  andACNFS  persistence 
out  to  7  days.  (February  through  June  2014) 


3.3.3  U.S.  Navy  Arctic  Submarine  Laboratory  Ice  Exercise  (ICEX) 

Navy  submarines  have  conducted  under-ice  operations  in  the  Arctic  regions  in  support  of  inter¬ 
fleet  transit,  training  cooperative  allied  engagements  and  operations  for  more  than  50  years. 
Since  1958,  the  U.S.  Submarine  Force  has  completed  more  than  120  Arctic  exercises.  U.S.  Navy 
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Arctic  Submarine  Laboratory  Ice  Exercise  (ICEX-2014)  began  on  Mareh  17,  2014  and  was  built 
into  an  iee  floe  north  of  Prudhoe  Bay,  Alaska  and  was  seheduled  to  continue  through  Mareh  30, 
2014.  However,  large  shifts  in  the  prevailing  wind  direetion  between  March  18*  and  March  20* 
created  instabilities  in  the  wind-driven  iee  floes  on  the  Arctic  Ocean  whieh  led  to  multiple 
fractures  in  the  iee  near  the  camp.  These  cracks  prevented  the  use  of  several  airfields  used  for 
transporting  personnel  and  equipment  to  the  ice  eamp.  The  rapidly  changing  conditions  of  the 
iee,  along  with  extremely  low  temperatures  and  poor  visibility  hampered  helieopter  operations 
and  made  sustaining  the  runway  potentially  risky.  Because  of  this  the  Commander,  Submarine 
Forces  (COMSUBFOR)  announeed  an  early  end  to  the  ICEX-2014  on  Mareh  23. 

Examining  the  ACNFS/NAVGEM  foreeasts  leading  up  to  the  events  show  that  large-scale 
fracturing  was  predicted  24  to  48  hours  prior  to  events  and  the  strong  winds  and  wind  reversals 
predieted  up  to  6  days  in  advance.  lee  fractures  were  already  prevalent  in  the  general  area  in  the 
weeks  leading  up  to  the  exercise,  but  strong  shifts  in  winds  brought  more.  Strong  easterly  winds 
picked  up  across  the  Beaufort  Mareh  15*  and  16*  and  a  strong  reversal  of  winds  oeeurred  Mareh 
20  to  2L*,  foreing  the  shut  down  of  operations. 

Figure  23  shows  the  increase  in  strong  easterly  winds  on  Mareh  15*  and  16*  reflected  in  the 
ACNFS  air  stress.  The  general  area  of  the  ice  camp  is  outlined  in  red. 
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Figure  23:  Beaufort  Sea  ACNFS  Air  stress  for  March  14,  15,  and  16,  2014.  ICEX  2014  ice  camp  general  region 
outlined  in  red. 


Figure  24  depiets  a  MODIS  and  RADARSAT-2  mosaic  image  for  Mareh  16*,  2014.  Fraeturing 
is  seen  throughout  the  region  with  fraetures  running  from  northwest  to  southeast  and  from  north 
to  south.  The  strong  fracturing  was  predicted  as  early  as  in  the  ACNFS  48  hour  forecast,  as  seen 
in  Figure  25. 
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Figure  24:  Beaufort  Sea  MODIS  and  RADARSAT-2  mosaic  image  for  March  16,  2014. 
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48  hour  forecast 


ACNFS-038  201^03-16  from  2014031218 
72  hour  forecast 
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Figure  25:  Beaufort  SeaACNFS  Opening  rate  nowcast  and  forecasts  for  March  16,  2014. 


The  strong  reversal  of  foreing  and  iee  velocity  occurred  March  20*  to  March  21®‘  and  was 
predicted  by  ACNFS/NAVGEM  as  early  as  the  6-day  forecast  as  shown  in  Figure  26. 
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Figure  26:  Beaufort  Sea  ACNFS  Ice  velocity  6-day  out  forecast  for  March  20,  2014. 


Some  central  Beaufort  openings  occurred  on  the  20*  and  21®‘,  but  strong  north  south  fracturing 
occurred  on  the  22"^*.  The  openings  on  March  22"‘*  were  forecasted  in  the  24  hour  forecast  as 
shown  in  Figure  27.  Figure  28  shows  a  MODIS  mosaic  showing  the  fractures  on  the  22"^  as  well 
as  wind  and  ice  drift  vectors. 
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Figure  27:  Beaufort  Sea  ACNFS  Opening  rate  hindcast,  nowcast,  and  24-hour  forecast. 
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Figure  28:  Beaufort  Sea  MODIS  mosaic  image  for  March  22,  2014.  Wind  and  ice  drift  vectors  overlaid. 
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3.4  Polynyas 

ACNFS  does  an  excellent  job  capturing  polynyas.  In  the  opening  rate  plots,  polynyas  are  well 
depicted  and  are  often  differentiated  from  fracture  areas  with  sharper  delineation  from  the 
surrounding  ice,  but  they  are  most  reliably  seen  with  the  ice  concentration.  Polynyas  are 
persistent  large  non-linear  shaped  regions  of  open  water  with  length  scales  on  the  order  of  100km 
and  generally  much  larger  area  than  ice  fractures  and  leads  (Martin,  2001).  With  their 
persistence  and  size,  they  are  typically  captured  in  satellite  ice  concentration  imagery  and 
consequently  the  model  ice  concentration  output.  There  are  two  types  of  polynyas:  coastal  and 
open  ocean.  Coastal  polynyas  are  found  adjacent  to  coastlines  and  are  driven  by  persistent 
offshore  winds.  Open  ocean  polynyas  are  enclosed  in  the  pack  ice  and  are  driven  by  warm  water 
upwelling. 


Figure  29  shows  an  example  of  an  open  water  polynya  seen  in  MODIS  imagery  and  ACNFS 
opening  rate,  and  ACNFS  concentration.  This  particular  polynya  persisted  for  over  a  month  until 
consumed  by  the  ice  edge. 


Figure  29:  Open  ocean  polynya  August  16,  2012.  Right:  MODIS  imagery  (NASA,  2014)  with  ACNFS  40% 
concentration  contour  overlaid  in  black.  Top,  left:  ACNFS  opening  rate  with  40%  concentration  contour.  Bottom, 
left:  ACNFS  ice  concentration. 


Figure  30  and  Figure  31  illustrate  an  example  of  coastal  polynyas  and  the  ACNFS  forecasting 
capability.  Figure  30  shows  8  consecutive  days  of  VIIRS  near-constant  contrast  (NCC)  imagery 
over  the  Kara  Sea  starting  April  15,  2014.  Note  the  polynyas  in  the  south  Kara  Sea  on  either  side 
of  the  Kara  Strait.  The  polynyas  fluctuate  in  size  as  the  ice  moves  away  and  towards  the  shores. 
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Figure  31  shows  the  nowcast  and  7-day  opening  rate  forecasts  from  April  15,  2014.  The  70%  ice 
concentration  for  each  model  forecast  is  contoured  and  overlaid  in  black.  Starting  from  the 
nowcast  -  day  April  15,  two  polynya  can  be  seen,  one  on  each  side  of  the  Kara  Strait.  The 
polynya  on  the  east  side  persists  throughout  the  time  period  increasing  and  decreasing  slightly  in 
size.  The  polynya  on  the  west  side  shrinks  and  then  returns  and  extends  northward  along  the  east 
Novaya  Zemlya  coastline.  These  same  features  are  seen  in  the  ACNFS  model  output. 


Figure  30:  VIIRS  Near-Constant  Contrast  (NCC)  imagery  (NRL-MRY,  2014)  over  Kara  Sea  from  April  15  through 
April  22,  2014. 


Figure  31:  ACNFS  Opening  rate  with  70%  ice  concentration  contour  in  black  for  Kara  Sea  from  April  15  through 
April  22,  2014. 
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3.5  Seasonal  Variation 

Seasonal  variability  in  the  performance  of  opening  rate  forecasts  is  seen  in  the  magnitudes  of  the 
RMS  error  and  classification  errors  sensitivity  and  specificity.  Figures  33  and  34  show  the 
seasonal/monthly  variability.  Forecasting  is  more  reliable  in  the  winter  than  in  the  spring,  but  at 
the  same  time  is  slightly  less  likely  to  predict  non-openings,  but  slightly  more  likely  to  predict 
openings.  The  accuracy  remains  consistent  between  seasons.  Forecast  specificities  were  lower 
in  February/March  as  the  number  of  ice  cells  reaches  a  maximum.  (Figure  34) 
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Figure  32:  RMS  Error  of  the  opening  rate  forecasts.  Winter  months  are  shown  in  shades  of  blue.  Spring  months  are 
shown  in  shades  of  green. 
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Figure  33:  Specificity,  Sensitivity,  and  Accuracy  classification  errors  of  opening  rate  forecasts.  Winter  months  are 
shown  in  shades  of  blue.  Spring  months  are  shown  in  shades  of  green. 
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Figure  34:  Daily  number  of  ice  cells  with  concentration  greater  than  15%. 


The  overall  accuracy  shows  little  difference  between  the  seasons,  but  the  sensitivity  and 
specificity  show  changes  especially  after  72  hours.  In  winter,  when  there  are  fewer  ice  openings, 
ACNFS  does  a  better  job  predicting  the  opening  regions,  whereas  in  the  spring  as  more 
fracturing  occurs,  the  model  does  a  better  job  predicting  the  non-openings. 


Table  1  of  Section  3.2.1  listed  the  category  totals  from  all  of  the  FLAP  message  comparisons. 
Table  3  lists  the  same  category  totals  from  all  the  FLAP  message  comparisons  against  ACNFS  by 
season/month.  As  with  the  model  forecast  to  model  hindcast  comparisons,  winter  had  overall 
better  results  than  spring  in  predicting  the  openings. 
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Winter:  Jan  -  Mar 

Fracture  Regions 

Percent  of  Fracture  Regions 

✓ 

? 

X 

✓ 

? 

? 

/  +  ? 

X 

off-set 

partial 

subset 

weak 

ACNFS 

2 

7 

0 

22% 

11% 

11% 

56% 

0% 

78% 

100% 

0% 

GOFS 

3 

6 

0 

33% 

22% 

11% 

33% 

0% 

67% 

100% 

0% 

ACNFS 
-  Accumulated 

4 

5 

0 

44% 

11% 

0% 

44% 

0% 

56% 

100% 

0% 

Spring:  Apr  -  Jun 

Fracture  Regions 

Percent  of  Fracture  Regions 

✓ 

? 

X 

✓ 

? 

? 

/  +  ? 

X 

off-set 

partial 

subset 

weak 

ACNFS 

15 

23 

9 

32% 

6% 

17% 

15% 

11% 

49% 

81% 

19% 

GOFS 

13 

24 

12 

27% 

2% 

14% 

18% 

14% 

49% 

76% 

24% 

ACNFS 
-  Accumulated 

16 

31 

1 

33% 

2% 

8% 

50% 

4% 

65% 

98% 

2% 

Summer:  Jul  -  Sep 

Fracture  Regions 

Percent  of  Fracture  Regions 

✓ 

? 

X 

✓ 

? 

? 

/  +  ? 

X 

off-set 

partial 

subset 

weak 

ACNFS 

6 

17 

3 

23% 

8% 

42% 

15% 

0% 

65% 

88% 

12% 

GOFS 

4 

14 

11 

14% 

0% 

34% 

14% 

0% 

48% 

62% 

38% 

ACNFS 
-  Accumulated 

9 

15 

3 

33% 

0% 

26% 

30% 

0% 

56% 

89% 

11% 

Fracture  Regions 

Percent  of  Fracture  Regions 

Fall:  Oct  -  Dec 

? 

X 

✓ 

? 

? 

/  +  ? 

X 

off-set 

partial 

subset 

weak 

ACNFS 

26 

42 

5 

35% 

3% 

18% 

24% 

12% 

57% 

92% 

8% 

GOFS 

22 

41 

10 

30% 

5% 

21% 

18% 

12% 

56% 

86% 

14% 

ACNFS 
-  Accumulated 

38 

44 

2 

45% 

4% 

7% 

37% 

5% 

52% 

97% 

2% 

Table  3:  FLAP  area  prediction  comparison  categories  totals  broken  out  by  season. 
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4  Summary  and  Recommendations 

This  validation  assessment  was  performed  using  ACNFS  with  some  limited  comparisons  with 
GOFS  3.1.  Results  of  the  validation  testing  of  the  ACNFS’  best  quality  prediction  against  actual 
FLAP  messages  as  ground  truth  and  model  forecasts  against  reference  hindcasts  were  presented. 
Additional  comparisons  were  shown  from  ICEX-2014  and  isolated  available  imagery.  Table  4 
presents  the  summary  statistics  for  the  NIC  FLAP  area  comparisons.  The  FLAP  areas  were 
strongly  matched  in  the  ACNFS  accumulated  openings  at  40%,  while  only  4%  were  missed 
completely.  Table  5  presents  the  summary  statistics  for  the  FLAP  forecast  capability  as 
compared  with  persistence  for  the  first  72  hours  of  forecasts.  The  ACNFS  forecast  showed  a 
47%  improvement  over  persistence  for  the  nowcast  in  terms  of  RMSE  error  with  an  18% 
improvement  out  to  3  days.  Accuracy,  sensitivity,  and  specificity  forecast  improvements  were 
approximately  30%  for  the  nowcast  and  20%  for  the  3  day  forecast. 


Percent  of  FLAP  Message  Fracture  Regions 

Strong  match 

Partial  match 

Strong  and 
Partial  match 

No  match 

ACNFS  accumulated  openings 

40% 

57% 

97% 

4% 

Table  4:  FLAP  area  skill  totals. 


Nowcast 

24  hour 

48  hour 

72  hour 

RMSE  ACNFS 

0.63 

0.82 

0.98 

1.09 

RMSE  Persistence 

1.19 

1.28 

1.3 

1.33 

Accuracy  ACNFS 

0.9 

0.86 

0.81 

0.75 

Accuracy  Persistence 

0.7 

0.64 

0.62 

0.62 

Sensitivity  ACNFS 

0.91 

0.85 

0.81 

0.78 

Sensitivity  Persistence 

0.73 

0.67 

0.64 

0.64 

Specificity  ACNFS 

0.88 

0.86 

0.77 

0.7 

Specificity  Persistence 

0.65 

0.6 

0.57 

0.58 

Table  5:  Flap  area  forecast  skill  metrics  for  first  72  hours. 


Overall,  ACNFS  provided  a  reasonable  prediction  of  the  openings  not  available  elsewhere  in 
either  nowcast  or  forecast  mode  and  performed  better  than  simple  persistence  of  the  best  model 
hindcast.  This  validation  has  shown  that  ACNFS  and  GOFS  3.1  nowcast/forecasts  provide  an 
indication  of  fracture  regions. 

Therefore,  it  is  recommended  for  the  National  Ice  Center  to  utilize  ACNFS  (and  GOFS  3.1,  when 
available)  opening  rate  nowcasts/forecasts  in  conjunction  with  available  imagery  to  better 
describe  a  current  realistic  sea  ice  scene  in  exercise/operations  planning  for  the  Navy. 
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7  Acronyms 

ACNFS 

Arctic  Cap  Nowcast/Forecast  System 

CICE 

Community  Ice  Code 

COMSUBFOR 

Commander,  Submarine  Forces 

DoD 

Department  of  Defense 

EVP 

Elastie  Viseous  Plastic 

FLAP 

Fractures/Leads  and  Polynyas 

GOFS 

Global  Ocean  Foreeast  System 

HYCOM 

HYbrid  Coordinate  Oeean  Model 

ICEX 

Ice  Exereise 

MODIS 

Moderate  Resolution  Imaging  Speetroradiometer 

NAVGEM 

NAVy  Global  Environmental  Model  (NAVGEM) 

NAVICECEN 

Naval  Ice  Center 

NAVOCEANO 

Naval  Oeeanographie  Office 

NCODA 

Navy  Coupled  Ocean  Data  Assimilation 

NIC 

National  lee  Center 

NOGAPS 

Navy  Operational  Global  Atmospheric  Prediction  System 

NRL 

Naval  Researeh  Laboratory 

OPEVAL 

Operational  Evaluation  (OPEVAL) 

RGPS 

RADARSAT  Geophysieal  Processor  System 

RMSE 

Root  Mean  Square  Error 

SSMES 

Special  Sensor  Microwave  Imager/Sounder 
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